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Abstract — Outer bounds on the admissible source region for 
broadcast channels with dependent sources are developed and 
used to prove capacity results for several classes of sources and 
channels. 

I. Introduction 

Consider a two-receiver broadcast channel (BC), say 
Pyz\x{')^ and a discrete memoryless source (5, T) with finite 
alphabet S xT. Source {S, T) is said to be admissible for this 
BC if for any A, < A < 1, and for large enough n there is a 
code with length-n codewords such that Pel < A and Pe2 < K 
where P^i and Pe2 are the respective error probabilities for 
receivers 1 and 2. The set of all admissible sources is called 
the admissible source region. 

Han and Costa [1] developed a coding strategy that admits 
certain sources. Let K — f{S) ~ g{T) be the common vari- 
able in the sense of Gacs and Korner (and also Witsenhausen), 
and consider auxiliary random variables W, U, V that satisfy 
the Markov chain property 



ST - WUV -X -YZ. 
Then the source {S,T) is admissible if (see [1], [2]) 



(1) 



H{S) < I{SWU;Y) - I{T;WU\S) (2) 
H{T) < I{TWV;Z) - I{S;WV\T) (3) 
H{ST) < mm{I{KW; Y), I{KW; Z)} + I{SU; Y\KW) 

+ I{TV;Z\KW) - I{SU;TV\KW) (4) 
H{ST) < I{SWU; Y) + I{TWV; Z) - I{SU; TV\KW) 
-I{ST]KW). (5) 

Our paper is concerned with "outer bounds" on the set of 
admissible sources, i.e., we wish to determine a superset of the 
admissible source region. To do so, we borrow tools from [3] 
and modify them to include dependent sources. We remark 
that a more general class of outer bounds for BCs is presented 
in [4]. Nevertheless, we use the approach of [3] because the 
auxiliary random variables are simpler and they can be used 
to prove capacity theorems. For example, it is not obvious 
how to choose the auxiliary random variables of [4] to even 
determine the capacity region of degraded broadcast channels. 



II. Outer Bounds on the Admissible Source Region 

Consider the source sequences S'™,T™ where m may be 
different than the code length n. Let R — n/m be the 
source-channel "bandwidth expansion". Further let the aux- 
iliary random variables (if, 5, T) have the same distribution 
as (if™, S'^.T'^) and let them form the Markov chain 

KSTUV -X -YZ. (6) 

We may also let X be a deterministic function of {S,T). We 
have the following outer bound. 

Theorem 1: An admissible source {S,T) satisfies the fol- 
lowing bounds for some Markov chain 

H{K)/R < mm{I{K; Y\U),I{K; Z\V)} (7) 

H{S)/R< I{S;Y\U) (8) 

H{T)/R< I{f;Z\V) (9) 

H{ST)/R < I{S; Y\fuV) + I{fU; Z\V) (10) 

H{ST)/R < I{f; Z\SUV) + I{SV;Y\U) (11) 

H{ST)/R < I{KUV;Y) 

+ I{S;Y\fuV) + I{f;Z\KUV) (12) 

H{ST)/R < I{KUV;Z) 

+ I{f;Z\SUV)+I{S;Y\KUV). (13) 
Remark 1: Consider the case R ~ 1, K,S,T statisti- 
cally independent, KST statistically independent of UV, and 
{H(K),H{S),H{T)) = {Ro,Ri,R2). The bounds ©-(HI 
are then a subset of those given in [3, Thm. 1]. 

Theorem 2: A weaker set of conditions than (l7]l- (fT3] l is as 
follows, where Q — {K, U, V): 

H{K)/R < min{/(Q; Y), I{Q; Z)} 

HiS)/R<I{QS; Y) 



H{T)/R<I{Qf-Z) 
H{ST)/R < I{S; Y\Qf) + I{Qf; Z) 
H{ST)/R < I{f- Z\QS) + I{QS- Y) 



H{ST)/R<I{Q;Y) 
H{ST)/R<I{Q;Z) 



I{S;Y\QT)+I{T;Z\Q) 



(14) 
(15) 
(16) 
(17) 
(18) 
(19) 



IiT;Z\QS)+IiS;Y\Q). (20) 
Remark 2: Consider the case described in Remark [T] The 
bounds (fT4l l-(l20li are then identical to those in [3, Thm. 3]. 



III. Examples 
We continue by considering only R = 1 for simplicity. 

A. Separating Source and Channel Coding 

A natural approach is to decouple ST from WUV, i.e., 
choose WUV independent of ST. Effectively, we convert the 
source strings to (compressed) bit strings and apply channel 
coding. The bounds (|2]i-(|5]i are then 

H{S)<I{WU;Y) (21) 

H{T) < I{WV; Z) (22) 
H{ST) < mm{I{W; Y), I{W; Z)} + I{U] Y\W) 

+ I{V]Z\W)-I{U;V\W)~I{S]T\K) (23) 
H{ST) < I{WU; Y) + I{WV; Z) - I{U; V\W) 

- I{S\T\K)^ I{ST;K). (24) 

B. Markov Sources 

Consider the case where S — K — T forms a Markov chain, 
that is I{S;T\K) — 0. This situation seems to be effectively 
the classic case where there are three independent messages 
Wo, Wi, W2 with uRq, nRi,nR2 bits, respectively. In fact, for 
any BC for which we know the capacity region, we can match 
the outer bound of Theorem |2] to the admissible bounds (l2Tl i- 
(|24] |. For example, consider a semi-deterministic BC where Y 
is a deterministic function of X. We have the following result. 

Theorem 3: A source for which S — K — T forms a Markov 
chain is admissible for a semi-deterministic BC where F is a 
deterministic function of X if ST satisfies 



H{K) < mm{I{W; Y), I{W; Z)} (25) 

H{S) < H{Y) (26) 

H{T) < I{WV; Z) (27) 

H(ST) < H{Y\WV) + I(yW; Z) (28) 



H{ST) < I{W; Y) + H{Y\WV) + I{V; Z\W) (29) 

for some Markov chain ST — WV — X — YZ. Conversely, 
such a source is not admissible for such a BC if for every 
Markov chain ST - WV - X - YZ the direction of one of 
the (strict) inequalities is reversed. 

Proof: For admissibility, choose U — Y and observe 
that the bounds (l25]l-(|29ll imply the bounds (l2T|-(|23l. For the 
converse, consider (fl4li-(fT7]l and ( fT9] l. identify Q,K,T with 
W, K, V, respectively, and use 

I{QS;Y) < H{Y) (30) 
I{S;Y\Qf) < H{Y\Qf). (31) 

■ 

C. More Capable BCs 

Theorem [3] does not give the admissible source region since 
it requires S — K — T lo form a Markov chain. In fact, Han 
and Costa showed in [1, Example 2] that separating source 
and channel coding (implied by decoupling ST and WUV) 
is suboptimal in general. The decoupling also doesn't seem 



to work for the expressions (|2]i-(|5]) for an important class of 
channels where one might guess that it should. 

Consider the class of more-capable BCs defined by the 
constraint that 

I{X; Y) > I{X- Z) for all Px- (32) 

This class includes both physically and stochastically degraded 
BCs. If we choose WUV independent of ST then (EB-lElll 
still exhibit the rate loss I{S; T\K) in (|23l)-(|24|. However, 
choosing W differently we have the following result. 

Theorem 4: The admissible source region for more-capable 



BCs is defined by the bounds 

H{ST)<I{X;Y) (33) 

H{T) < I{W; Z) (34) 

H{ST) <I{X]Y\W)+l(W]Z) (35) 

for some Mai'kov chain ST -W - X - YZ. 
Proof: We choose 

W = TW where ST is independent of WUV (36) 

U = X — s. noisy function of W (37) 

F = a constant. (38) 



The bounds (l2]i-(|5]l are thensimply (|33]|-(|35]|. Furthermore, for 
any Markov chain ST — W — X — YZ we can achieve the 
right-hand^sides of d33Tl-(l35Tl since these depend on ST only 
through W . For the converse, consider (fT6]l-(fr8Tl and identify 
QT, S with W, U, respectively. The bounds (I33ll-(l35]l follow 
from ([TSTl-lfTsTl and the bounds 

I(T;Z\QS) < I{X-Z\QS) 

= PQg{ab)I{X; Z\QS = ah) 

^I{X;Y\QS) (39) 

where the second inequality follows by applying ( [32] l. ■ 
We remark that the choice W — TW does, in^ct, 
permit separating source and channel coding because W is 
independent of T. The coding approach is to simply compress 
T to its entropy-rate H{T) and consider the resulting bits 
as a common message. Next, compress S to the conditional 
entropy-rate H{S\T) and consider the resulting bits as a 
private message. Decoder Y first decodes and decompresses 
T and then decodes and decompresses S. Decoder Z decodes 
and decompresses T only. This natural approach is included 
in the Han-Costa coding method, but only indirectly. 

IV. Proof of Theorem[T] 

Let \S\ be the cardinaHty of the set S and let (K, S,f) = 
{K'^,S'"\T"^). Fano's inequality [5, Sec. 2.11] gives 

< < Pel • r7ilog2 \S\ + 1 (40) 

< H{T"'\Z'') < Pe2 ■ mlog2 \T\ + 1 (41) 



Let 6i — Pel log2 |iS| + l/m and 62 — Pe2 log2 |5| + 1/to, and 
observe that reliable communication {P^i and Pe2 0) 
means that (5i — > and ^2 as to — > 00. We define the 
following auxiliary random variables 



Pi{i) = i = l,2,...,n 

n 

U = {Uj,I), V^{Vj,I) 
X = Xi, Y = Yi, Zi = Z 

and observe that we have the Markov chain 

KSfuV -X -YZ. 



(42) 
(43) 

(44) 
(45) 

(46) 



Furthermore, the definition ( |45] ) of XYZ is consistent with our 
BC in the sense that Pyz\x{') is the same in Sec. J] Fano's 
inequality ( |40l ) implies 

m[H{K) - 61] < H{K"') - H{K"'\Y'') (47) 
= /(ii:™;r") (48) 

n 

= ^I{K'^;Y,\Y'-^) (49) 
1=1 

n 

= ^/(i^;y,|f/0 (50) 

1=1 

= nI{K;Yi\UiI) (51) 

= nI{K;Y\U) (52) 

We similarly have 

m[H{K) - 62] < nI{K; Z\V) (53) 

TO[ff (S*) - (5i] < n/(S'; Y\U) (54) 

TO[i7(r) - ^2] < nl{f; Z\V). (55) 

To develop our other bounds, we will use the following 
identities (see [6, p. 332] and [7, Lemma 7]). 

Lemma 1: For any random variables W,Y", Z" we have 



n 

I{W; Z") = J2 [l{WY'-^;Z';) - I{WY'; Zf+i) 



(56) 



where Y 



-'n+l 



0. 



Proof: By direct calculation. ■ 
Lemma 2: For any random variables W,Y", Z" we have 

n n 

J2l{ZuY^''\WZ^,) = Y,I{Yf,Zl^,\WY^-'). (57) 

1=1 1=1 
Proof: See [7, Lemma 7]. ■ 

Consider the following steps: 

m[H{S) + H{T) -Si- 62] 



< I{S"';T"') + I{S"';Y''\T"') + /(T"; Z 

n 

= mI{S;T)+Y^ [/(5";y,|T'"y*-i) 

i=l 

+ liT'^Y'^^; Z^) - liT^'Y'-Z^j^^) 



(58) 
(59) 
(60) 



(61) 



where the last step follows from Lemma [T] Continuing, we 
have 

m[H{ST) 82] (62) 

= \^I{S"';Y,\T"'Y'-^) + liT'^Y'-^; Z,\Z'^+i) 



n 

[ - |r™ s-^r'-i) + i{t"'y'-^;Z,\zi'+^) 



+ H{Y,\T'^Y'-'-Z2+i) 

n 

< ^ [ - H{Y,\T"'S"'Y'-^Zl'^^) 
1=1 

+ /(T^r'-i; Z,;|Zf_^i) + H(Y,\T"^Y''-^Z^_^^) 

n 

- Y [l{S;Y,\fu,V^)+I{fUf,Z,\V^) 

i=l 

= n\j{S;Y\fuV) + I(TU]Z\V) . 
Similar arguments give 

m[H{ST) -5i- 82] < n \l{f; Z\SUV) + I{SV; Y\U) 



(63) 

(64) 

(65) 
(66) 
(67) 



(68) 



Next, consider the following steps: 



m[H{S) + H{T) -61- 62] (69) 

< /(S""'; y") + /(T"; Z") (70) 

< /(if"; r") + /(S""; r"|if") 

+ I{K"'-T"') + I{r"';Z"\K"') (71) 

< +/(S'™;T"|if") +/(5"';y"|if™r™) 
+ /(if"; T™) + i(r"; Z"|if") 
±/(T™;r"|/f'") (72) 

= /(/f™5"r™; F") - /(T"; F"|/f'") + /(T"*; Z"|/f™) 
+ TO(/(5;T|if) + //(/i:)). (73) 



The reader can check that 

I{S;T\K) + H{K) ^ I{S-T). 
The first expression in jTSl l is bounded as 



(74) 



I{K"'S"'T"'; r") = ^ I{K"'S"'T''; Y,\Y'-^) (75) 

i=l 
n 

= ^/(if5f;y,|C/,) (76) 

i=l 
n 

<Yl(KSfu^V,;Y{) ill) 



The second and third expressions in ( |73] i can be manipulated 
as follows: 

~I{T'";Y"\K"')+I{T'^;Z''\K"') (78) 

n 

= ^ [-/(r'";y,|if"y*-i) + /(r™;Z,|if™z;Vi) 



(79) 



n 

1=1 

+ I{ZlXi;Yi\K"'T"'Y'-'^) 
+ I{T"'Y'-^;Z,\K"'Z^_^^) 

- I{Y'-^;Z,\K"'T"'Zl'^^) 

n 



/(T"y*-i;Z,|ii:'",z,'Yi) 



(80) 



(81) 



where the last step follows from Lemma |2l Continuing, we 
have 



/(T";y"|X") +/(r";Z"|if") 

n 

1=1 

- I{T"'-Y,\K"'Y'-^ , Zl\^) 
+ I{Y'''-Z,\K^Z:t+i) 
+ I{T"';Zi\K"'Z^_^^Y'-^) 

71 

[ - /(T";y,|x'"r'-izf+i) 

i=l 

+ I{T"'-Zi\K"'Z2_^^Y'-^) 

n 



(82) 



j=l 



(83) 

(84) 
(85) 



where the second last step follows from Lemma |2] 
We substitute CD. and into and obtain 

m[H{ST) -Si- 62] (86) 

n 



+ /(T;Z,|i^;7,F,) 

n 

^ [/(i^f/.F,; K,) + I{S; Y,\kfu,Vi) 

i=l 

+ i{f-z,\ku,Vi) 

n \l{kUiVi-Yi\I) + I{S- Yi\fUiViI) 
+I{f-Zi\kUiViI) 



(87) 



(88) 



(89) 



<n I{KUV;Y) + I{S;Y\TUV) + I{T;Z\KUV) (90) 



By similar arguments, we also have 

m[H{ST) -Si- 62] 



< n 



I{KUV; Z) + I{T; Z\SUV) + I{S; Y\KUV) 
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